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Overview pf t Limuisttc Theory of DesitV 
Hark L. Bill or and Ira P. Gold it e In 

The SPADE theory uses linguistic formalist!* to model the program planning 
and debugging processes, The theory has been applied to constructing a 
grammar-based editor in which pro grama are written in a structured fashion, 
designing an automatic programing system based on an Augmented Transition 
Nettror% and pining protocols of programing episodes. 

The SPADE theory begins with t taxonomy of basic planning concepts, 
covering strategies Tor identification, dacQmpn5.it ion and r* fcrmu la t ion r A 
handle is provided for recognizing interactions between goal* and deriving a 
linear solution. 

A conplementary theory of rational bugs and associated repair techniques 
la also provided, SPADE introduces a new data structure to facilitate 
debugging — the derivation tree of the program. The SPADE editor makes thia 
Structure aval labia ta th.fi p ro £| riiama r . 

The SPADE theory generalizes recant work in Artificial Intelligence by 
Sus&man and Sacerdotl on automatic programming, and extends the theory of 
program ctesign developed toy the Structured Programming movement. It provides 
a more structured information processing nodal of hunan problem solving than 
th* production system* of Newell and Simon, and articulates the type of 
problem solving curriculum advocated toy Ptpert's Logo Project. 

This report describes research done at the Artificial Intelligence Laboratory of 
tne nassacnu setts Institute of Technology, The automatic p rob lea solving aspect of 
this research was supported by the Advanced Research Projects Agsncy of the Department 
Of Defense under Office of Naval Resaarch contract NQD0H-7VC-O643, the educational 
aspect by the National Science Foundation under grant C4e7QBX> and the protocol 
analysis aspect hy the Intelligent Instructional Systems Group at Bolt Beranek and 
Nnwnen, under contract number PIDA 903-7&-C-Q lOfl Jointly sponsored by Advanced Research 
Projects Agency, Air Force Human Resources Laboratory* Army Research Institute, and 
Naval Personnel Research & Development Center. 

] Ttils is en expanded version of a paper submitted to the fifth International Joint 
Conference on. Artificial Intelligence. 
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I. A Multi-Fa ce ted Approach 

This paper provides an overview of three separate contexts in which the SPADE 
theory is being simultaneously developed and tested -- conputer uses in education. 
automatic programing <i traditional A l arena J, and protocol analysis (the domain of 
Information processing psycholooy). Our experience has been that a powerful jynerglsm 
results from this multi-faceted approach. 

1. The Education Contest: an editor has been implemented that encourage* students 
to define and debug program in terms of explicit SPADE design choices, rha editor 
provides a structured programming envihOhnant based on a detailed theory of the 
processes Involved in coherently Structured problem solving, 

2. The Ai Content : an automatic programmer celled PAIN has been designed using an 
augmented transition network embodiment or tha SPADE theory. This results in a unified 
franework which clarifies recent WOrlt oh planninfl and debugging by Sacerdotl [19/5] and 
Sussman [1975J. 

3. The Psychology Context: a parser called PAZATN has been designed that applies 

the SPADE theory to the analysis of programming protocols. PAZATN produces a parse of 
the protocol that delineates the planning and debugging strategies employed by the 
problem solver, PAZATN extends the series of automatic protocol analyzers developed at 
Cot-tiegle-Hollon University [Vattrman It Newell 1975, 1M4: Dtiaskar ft Simon 1&76). 

Hand-simulations of PATW and PAZATN on elementary programming problems and 
informal experiments with the SPADE editor attest to the theory's cogency in accounting 
fOr ■ wide range of planning and debugging techniques [Goldstein ft Fliller 1976a, b; 
Miller ft Goldstein l976b,c,d]* 

2. A Linguistic Analog y 

In developing a r a pro s nsi 1 1 on Tor problem solving techniques, we have been tftllded 
by an analogy to computational linguistics, for threa reasons. 

1. The concepts and algorithms of computational linguistics* though originally 



intended to explain the nature of language per ie, supply perspicuous yet powerful 
descriptions of complex computations In general. Problem solving, as ■ complex 
process, benefits from the application of these tools - 

Z* Computational linguistics decomposes computations Into syntactic, semantic, and 
pragmatic components, This decoiapasitk.Dn clarifies the explanation or complex. 
processes whEn viewed In the following manner; syntax formalizes the rango of possible 

decisions; semantics the problem description, and pragmatics, the procedural 
relationship between the two. 

3. Computational linguistics has undergone an evolution of prncHdurnl formalisms, 
beginning with finite state automat a H later enploying recursive transition net works 
(context free nrammars ) h next moving on to augmented transition network,*, and 
culminating in the current set oT theories Involving frames [ttlnsfcv 1475. 
Uinograd 1975, Schank. 1975}. Each phase captured some properties of language, but was 
incomplete and required generalization to sore powerful and elaborate formalisms * 
Pol lowing this evolutionary sequence illuminates the complexity of language theory, k> 
have pursued a similar evolutionary approach, to clarify the complexity of problem 
solving processes. 

To date, our theory of design has evolved through the following stages: w* 
Initially explored content free grammars for planning and debugging, and subsequently 
their generalization to ATN's; we examined the metaphor of protocol analysis as 
parsing, initially using the planning and debugging grammars to reveal the constituent 
Structure of protocols and later using the derivations produced by the ATM formalism; 
and, finally, our most recent work, has studded the use of a chart-based parser to 
discover these analyses. 

3, ft G ramnat 1 c a 1 Theory of Planning 

The basis for SPADE Is a taxonomy of frequently observed planning concepts 
(Fig. 1). Ve arrived at this taxonomy by introspection, by examining problem solving 
protocols [Miller h Goldstein 1976b], by studying the literature on problem solving 

[Polya 1957, 1965, 1968; Newell ft Simon 1972; Sussman 1975j Eacerdoti 1975 J, and by 
iratlnoj techniques for finding procedural solutions to problems expressed as 
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predicate calculus formulae [Enden * Kowalski 1916]. This last criterion demonstrate* 
that the taxonomy is currently inconplete -- for exanple. techniques Tor handling 
disjunctions have not yet been analyzed thoroughly enough to warrant inclusion. 
However, tha taxonomy Is adequate for I wide range of Blenintiry programming problems. 
Future resfiarch will Investigate additional planninfl techniques. 

There are three major classes ef plans in the ta^enepy; Identification. 
decomposition, and rer«rnulaticn. Identification Deans recognizing a problem as 
previously solved. Decomposition referi to strategies far dividing a problem into 
simpler sub -problems.. Refornulatiaii plans niter tM problem description, seeking a 

representation which is mere amenable to identification or decompaction. The figure 
indicates how these classes of plans arc Turther subdivided in the SPADE tneory. 

Planning, according to the theory, is a process in which the problem solver 
selects the appropriate plan type, and then carries out the subgoals deflnetl by that 
plan applied to the current problem. From this viewpoint, the planning taxonomy 
represents a decision tree or alternative plan. The decision process can be modeled 
by the content free grammar given below. The grannar explicitly States which planning 
rules involve recursive application of solution techniques to subgoals: setup, 
interface, da in step, cleanup, and parallel. 

(The rules flf the grammar are written using the following syntax: "I" is 
disjunction, * + ■ is ordered conjunction, "ft" Is unordered conjunction, 
■<,..> ■ li iteration, [,..] It Optionality, and a lower case English phrase 
enclosed in quotation marks (en,,, ■repeat step") describes a lexical item 
which is not further expanded In the grammar,) 

PLAN -> IDENTIFY I DEC0PIP05E | REFORMULATE 

IDENTIFY -> PRIMITIVE | PEFLNtD 

DEFINED -> -call user subprocedure" ft P LAM 

DECOMPOSE. -> CONJUNCT 1 0*1 I REP ET I HON 

CONJUNCTION -'} SEQUENTIAL | PARALLEL 

SEQUENTIAL -> (SETUP] # <h*A INSTEP + (INTERFACE ] >* + [CLEAJFUP] 

PARALLEL -> <PLAN>* 

SETUP -> PLAN 



HA INSTEP 

INTERFACE ' 

CLEANUP 

REPETITION 

ROUND 

ITER-PLAN 

TAIL -RECUR 



~> PLAN 
-> PLAN 
-> PLAN 

-> ROUND I RECURSION 

-> ITER- PLAN | TAIL -RECUR 

-> "repeat step" + 5EQ 

-> "stop step - * SEQ * "recursion step" 



The SPA&E theory Is not restricted tQ any particular domain. However, to provide 
concrete exdnple& h we have concentrated on problems froo elementary Loao graphics 
programming [Papert 1371], This domain Kas chosen because of the availability of 
extensive data on the performance of students writing *turtle" programs to draw sispi* 
pictures, The grammar rules for primitives in this denain are: 



PRIHITIVE 

VECTOR 

ROTATION 

PEN3TATE 



■> VECTOR | ROTATION 1 PEN5TATE 

O [FORWARD | BACH] + <nua*er> 

-> (LEFT I RIGHT] + < number? 

■> PINUP | PENDOWN 



A typical task undertaken by beginners In the Lego environment is to draw a wishingwali 
picture uaing the computer . 




Fig, Z — A Lego UishlhtfHell 
Fipj. 3 Illustrates l solution to the wlshingwell problem with its hierarchical 

annota t ion according to our planning griiima r . 



The gramaar characterizes the decision process Involved in selecting plana fr™ 
the taxonomy, We illustrate its utility In the nest two sections by constructing 1 an 
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editor that embodies the graranar and analyzing debugging in t*™s Pf the gran^r. Then 
WB sho* how the srammar can t« Hignwited f Include not only th B synte* of Plan*, hut 
-.heir saiaantics and pragmatics as well. 

4. s?ftD E - & , A p lati n lng_AsslsU n t 

On* reason, for calling o»r theary of planning and debugging itrucEnree' 1ft to 
enphasize the link b«t«e*n our raseirch and the Structured Primming aovnient, DahL, 
Dijkstrft. and Hoare [197Z] call for a stvle of proaranlmi which reflects coherently 
structured P ro&i« solving, But a detailed formalization of *Jiat tMi style entaili ii 
lacking. Our efforts In this direction, therafore, supplant the work of Dijkstra and 
other** The spAOE-0 editing environment serves as an example. 

Hew ctn wb judge whether a particular graawr uf plana captures, at sone level of 
abstraction, the set of planning decision* involved in solving problems for ao» 
donain? One nethodology *e employ i* to incorporate the grannar Into an editor (SPADE- 
0) *hose purpose is to auoaent and direct the capabilities of a human user, The 
critical question then becones determining tha extent to which such a s*st™ aide nr 
hinder? the user- 

Suppgse a prehlem solver is defining a Logo program for drawing the Hlshtnoirtll 
shown earlier, in SrADE-D, this is accomplished oy applying the planning gra-iar in 

aetiftrative nrnda: 

la. What is the nana of your top lave! procedure? 
lb. >UW 

2a. The ml a is: PLAli -> IDENTlFf I DECOhmt I ftETORfflJLATE 

What now? 
2b. >» ECO* POSE, 

3a. The rula IS; DtCOhPqSE -> CONJUNCTION | REPETITION 

3b + JCOMJ0NCTI0N 

4*. The rule Is; CONJUNCT I ON -> 5ECHJEHTIAL. 



Tha rule is: SEQUENTIAL -> {SETUP J + <F1A1NSTEP ■* [INTERFACE]/ * [CLEANUP] 
fin you wish to define an nptiann.1 SETUP7 
4b . Later . 



Da _ . _ _ 

In this Way, SPADE-0 encourages users to articulate their design decisions, In top- 
down ardor, At the same tine, the system allocs the user to escape From this strict 
discipline if an alternative solution order seons. proferable. This was illustrated by 
the. user's "later' in Struct ion. "Later' suspends the current goal for later solution. 

SPADE -0 was implemented by assigning an interpretive procedure to each grammatical 
operator. In essence, the editor is a bookkeeper for the User's goal tree. Despite 
its simplicity, mo see threo useful applications of the program. 

1. From an educational standpoint, the editor encourages students to articulate 
their problem solving strategies. The fundamental hypothesis of the Loijo Project, as 
prosentod by Papert in reocAftfj CHildrvn Thinking [1971], is that such articulate 
problem solving is beneficial to the learner. The SPADE editor, with Us extreme form 
of articulation, provides an experimental vehicle for evaluating Papert 1 s claim: do 
students exposed to SPADE progress Faster in Learning Logo thin controls whoso problem 
salving is more tacit? 

Z. From an A I standpoint, its use will indicate whether the planning gnumer It 
adequate, or whether certain plans are not present that competent problem solvers feel 
■re necessary. 

1. From a psychological standpoint, we will collect transcripts of Individuals 
using the editor and formulate personal are/mars based on the particular rules usually 
employed by each user, The personal grammar will model the problem solving skills of 
that individual. In the past we have manually analyzed prntocols from standard Logo, 
SPADE pro toco is , with their explicit planning choices, can be far more revealing. 



3. RAID, A Debugg i ng Assistant 

SPADE includes t theory of debugging. Such * theory is essential, since problem 
solvers nust ortan formulate plans in the ftce OT inperTect Knowledge and limited 
resources. Under such Clreunstancas, even carefully reasoned Judgments may be 
mistaken. 

Given 9 grammatical theory of planning, debugging can be analyzed as the 
localization and repair or errors in apolylng grammar rules during planning. Since our 
planning rules here constructed Iron operators Tnr conjunction, disjunction, and 
optional lty, three basic classes of errors arise: 

1. syntactic bugs. In which the basic grammar is violated, such as when a 
required conjunct is missing; 

2+ semantic hugsi In which a semantic constraint arising fro* the particular 
problem is violated, such as when 4 syntactically Qptiattal constituent, needed 
because Of the semantics of the particular problem, is missing; 

A, pragmatic bugs, in which an inappropriate selection from a set of mutually 
exclusive dta/biflcts IS made. 

Fig. 4 provides examples of these bug types. Although additional categories 
(Including execution and efficiency bugs) must be defined tg make this taxonomy of bugs 
CObplete. these Classes are adequate to characterize many examples which arise in 
elementary programming, 

RAID is a debugging assistant for SPADE-O implemented oh the basis of this 
taxonomy of errors, As 3FA&E-Q prompts the user through planning process* those 
choices which might lead to syntactic, semantic, or pragmatic bugs are recorded by the 
system. In subsequent d-ebUdging episode*, PAID alerts the user to the relevant bug 
warnings, thereby aiding in diagnosis, Moreover, the spade -d planning editor can than 
be re invoked to guide repair of the &U*l by resuming planning In the context of the 
culpable decision, An interaction with RAID thit would arise from premature execution 
oT a program Is Illustrated be low; 



FIGURE A - SYNTACTICALLY INCORRECT PLAN 

A NECESSARY CONJUNCT IS MISS 1 N<1 



TO WW 

10 TRIANGLE USE 



END 



WW 

??? TRIANGLE UNDEFINED ??? 



ID-PLAN 



GUT 



(■GET" MISSING. UNGRAMMATICAL NAN 
DEBUG BY COMPLETING FLAW, ) 



GET TRIANGLE FILE 



FIGURE 4 - EEMANTICALLV INCORRECT TLAN 
AN OPTIONAL CONJUNCT IE HISSING 

FOR EXAMPLE: "WW" MISSING INITIAL SETUP, AND INTERFACE FOR lOLF: 



TO WW 

ID WELL- — MAINSTEF- 

20 POLE — HA INSTEP- 



t> 



L StQ-i'l*Mi 



ends hero 



t 



J 



strirtfi horo 



FIGURE 1 - PRAGMATICALLY INCORRECT I' LAN 
AN INCORRECT DISJUNCT HAS BEEN SELECTED 



TO SQUARE-INSlDE-TRIANCfLE 
10 SQUARE 
20 TRIANGLE 
END 

INTENDED PICTURE: 



LINEAR rLAN -- 

sou arc; and triangle 

DESIGNED 
INDEPENDENTLY. 

ACTUAL. PICTURE I 





TH* (tier has ceirtiflired de/tnlflfl tft* aist\ia$mli provrvn bwyun mcrller aith 
its caffcjrt definition as shean 1 .n /Iff. tfe, 4ftftotlff& 9W initial StTVP an* dA 
INTERFACE ftv&t not J/«t 0Mff tupplivA, tAs ajar refills eiEcut ion of the- 

program. 

la. ... Miat now? 
lb, >RLll WrSHlNGWELL. 

2ft, OK. <Tf f sr 4b shotsti t/ir remLtinj picture,) 

What now? 
Zt». >[HaHno5Q 

la. The rule Is: DIAGNOSE-) <PARSE | COPE I MODEL I PROCESS)*. 

What nowr 
3b . >Parse 

4e, The rult i»: PARSE -> SYNTAX I SEMANTICS I PRAfiNATJC3. 
What now? 

4a, >£emantici 

5ft, The following optional constituents ire nlising in WW: 

Til* SETUP and tho INTERFACE between WELL and POLE. 

What now 7 
5b. >Reptlr 

6i. The nil* la: REPAIR -> COMPLETE | REPLACE. 
6b, >ConplBtfl Interface between well end pelt. 

7a. The rule is; PLAM -> IDENTIFY I DECOMPOSE I REFORMULATE 
7b. > ttt 

In thia example, the Interaction with SPADE wis again guided by a gr enter; thli 
debugging granmar is given below. 



DIAGNOSE O <PAK5E | CODE I MODEL I PROCESS* 

PROCESS -> <ASK| TRACE >* 

CODE -> "print definition* 

MODEL -> RIW 

PARSE ^> STNTAX I SEMANTICS I PRAGMATICS 

SYNTAX -> "review conjunctive choices in plan - 

SEMANTICS -> "review option*! choice* in plan" 

PRAGMATICS -> "review disjunctive en dices In plan" 

REPAIR -> COMPLETE I CORRECT 

COMPLETE -> ■solve for missing conjunct* 

CORRECT -> "choose altarnativB disjunct* 

RUN -> "run code line" + ["error nsg"] + [DEBUG] 

ASK -> "print definition" | 'print value" 

TRACE -> [SELF- DOC*] + RUN* 

SELF -DOC -> 'add breaV | "add print" | "idd (rue" 

This, grammar represents the traditional debugging strategies enployed by programners 
supplemented by the PARSE-basfld strategies allowed by the existence uf the SPADF. 
derivation tree. 

For mora complex programs, there Mill be too many potential bug locations for 
RAID's undirected advice to be helpful. Hence, following SPADE-O's structured 
bookkeeping philosophy, we allow the User to set caveats during planning regarding 
those decisions which appear to he potential trouble spots. In this fashion* SPADE 
permits an exploratory style of problem solving, as illustrated below. 

Suppose the wishingnell proQtart is fisrl 0/ C tvrQt aro/ect. Then the number 
0/ optional cons ti tuenta that have not hrcn deftucd will Jw large- The VS*T 
uilt probably kittte some lasight Into wMcJt 0/ these is cclv.vUy needet, To 
.facilitate dtbujfsi its 1 the user is entQur<rg*<t to place caveats at tAose 

decisions he suspect* rtau rrfftifre idler ri>:>,K\ci i •$ 

6a. Do you wish to include the optional initial SETUP in WW? 



7a. Dp you wish to retard a caveat here 7 
7b. >Yes> 

fta. QE t four pending goals are HOOP, PULE and WELL. What now? 
&b. > ... 

Another mechanism to nake SPAbl's and ftA£D"s advice more directed it to supply til* 
sflMnticj for problems. This is the topic of the next jectlon. 

6, PATH " An Augmented Transition network for Planning 

While context: free grammars can represent a useful abstraction of planning 
decisions, they have limitation* which prevent the* from providing i Complete theory of 
design. They provide no representation for the semantics or the problem nor for the 
pragmatics involved in choosing one- plan over another. For this reason. w# have 
designed and are currently implementing FATN. an augmented transition network (ATN) 
problem solver. we have adopted the ATM formalism Tor the Same reasons that led Hoods 
C1970] to introduce it into computational linguistics: the semantic and pragmatic 
limitations of context free grammars. 

Fig. S provides a global view of PATH [Goldstein & Hiller LSJobl, The topology of 
the network embodies tha planning grammar, Registers Contain descriptions of the 
problem, the solution, and various temporary constructs built during planning. Arc 

predicates supply pragmatic guidance by examining 1 tha registers and appropriately 
directing the planning process. For example, an identification plan cannot proceed If 
the problami description cannot ba found in tha answer library. PATH has been 
successfully hand-simulated on elementary Logo and Blocks World problem, 

PATH allows us to elaborate our notion of a completed plan by defining in 
annotated derivation tree. Associated with each node of the plan derivation Is a 
snapshot of the values of the ATH registers at the point in the planning process whan 
that node was created. A derivation trea reveals the constituent structure of the 
plan; these semantic uprleeJej reveal the semantic Intent, A set of Gtsis-Honi at 
each node, derived as instances of PATH'S arc predicates, reveal the prapmatLc reasons 
far preferring a given plan over Its competitors. Fig. 6 shows the annotated 
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derivation tree for th6 *flshintPfell program given earlier. 

An intended application or PATU is as a module of the SPADE editor for providing 
the user with Additional direction And advice, The following hypothetical dialogue 
With SPADE- 1 {the SPADE editor augmented by PATN) Illustrates this, 

■ 

la. Tell mo the name of your tep level procedure T 
lb. >VV 

2b. Would you lite to define a model* 

» 

2b. >W I SHI WGWE Li- 
sa. OK, WISHI^ICUELL medal found in Model Library, tno rule is: 

PLAN -> IDENTIFY | DECOMPOSE I AEFQRHULATEh Whet now? 
3b. >ROOF POLE WELL 

4a L These SUDQGals correspond to model parts. 

Are you pursuing a LINEAR DECOflPOSlTlOli plan 7 
4b. >VES 

S*. Fhe pending aubgoals are: ROOF^ POLE, WELL end the preparatory steps. 

What now? 
*b. ?RWF 

fla. In the WW ncdel> ROOF is described as a triangle. 

There Is a TRIANGLE procedure in the answer library. 

Would you like to use it? 
e*. >YE5 

7a. OR. The ROOF is complete. Tour pending gotls are: 
POLE, WELL, and the preparatory Steps. What now? 

7b . > ... 

PATH'S Linearization Loop provides a theoretical superstructure fer handling 
dependent, interacting suhgoals. The approach Is to view a non-linearity a; a property 



or the .problem description* That is. even though oni description of the problem loads 
to an unsuccessful refinement process because of Interactions between subgoals. another 
description can succeed by flaking the dop§ndent constraints explicit. For example, 
consider the SQUARE- INS I DE-TMANGL£ or fig. «. If the problem description is; 

SQUARE X; 
TRIANGLE Y; 

INSTDE X Y; 

then e decomposition that draws the square and triangle independently and then attempt! 
to fit then together to achieve the inside relation will fell. However, a problan 
description of the following fora allows a successful decomposition; 

SQUARE X, WITH SIDE = 100; 
TRIANGLE Y, V1TH SIDE > 100; 
CENTER OF X ■ CENTER Of Y. 

The INTERACTIONS predicate is a conjunction of tests on the model register. Each 
test is responsible for detecting a given non- linearity. A corresponding action 
modifies the model, adding new statements to make the interaction explicit. The 
REFINEMENT loop is the repository for what Sutsman [1975] calls the Critics Gallery* 
The theoretical progress of PAT.V is to integrate the Critics Gallery concept Intft A 
theory of planning, in Su&sman's HACKER h the critics gallery and library or 
programming techniques were separate modules: there Was no integrated theory. 

Of course, at any point In time the system nay be unaware of a given type of non- 
linearity. In such cftses, the absehte of an Interaction test will lead to a sequential 
decomposition that ultimately falls. The design of a program Tor debugging such 
failures is the subject of the neitt section. 

T, PAPR -- Aft Augmented Tra ns ition Wetworfc for Debugging 

PATH Can make mistakes. That is, PATN will sometimes Introduce what we term 
rotioant bwgs into Us plans, due to asking arc transitions with imperfect knowledge of 
subtleties or inter act ions in the task domain. Hence, PATN must b-e equipped with e 



coapiaiientary debugging module, DAPR (rig. 7). 

While RAID was designed to assist human prograianers In finding a variety Of bugs 
(primarily by plan diagnosis), DAPR has designed to analyze PATN's bugs. DAPR employs 
three diagnostic techniques: model, process h and plan diagnosis, Model diagnosis is 
the basic technique. U amounts to comparing the effects of executing i plan to a 
formal description of Its goals, to determine If, and In whit fashion, the plan has 
failed. Another DAPR technique, based on Sussman's HAfTFR [1975], is Examining the 
state of the process at the time of the error Hani testation, plan diagnosis is a DAPS 
first. It is accomplished by examining the attests variable Associated with various 
nodes of PATM 1 s annotated plan derivation. 

DAPR will also be used to provide additional guidance to RAID. This illustrates 
the synergism possible when educational, psychological and At facets of a cognitive 
theory are Studied ip an integrated Fashion. This Integration is. further exemplified 
in the noKt section when we apply the SPADE theory to protocol analysis. 

■ 

8. PAZftra, an A utocra tic Protocol Ana l yzor 

As soon as one has an heuri slice! ly adequate theory of design, it is natural to 
ask, "Can the theory provide an account of how people solve problems?"'. An 
experimentel technique we employ for answering this Question is th* analysis df 
protocols collected during problem solving sessions. By adopting this nethodnlogy we 
follow the precedent established in sanltial protocol analysis studies conducted at 
Carnegie Nelloh University [ Hawaii fc Simon 197Z; waterman ft Heitfell 197Z, 1973; 
Bhaskar ft Simon 19/76 J . Our work extends their approich along three dimensions. 

1. With the exception of the recent Bhaskar ft Simon effort, the CMU studies have 
been restricted to very limited domains such as erypcaritfcmetic. Rather than limiting 
the task domain, we limit the range of responses. Typically protocols are 
transcriptions or thint-aloud verbalizations; we focus on the more restricted 
interactions arising from a problem solving session at a computer console. The 
Analysis test in this setting li to Interpret user actions — editing, executing* 
tracing, etc. -- in terms of the SPADE theory of planning and debugging. 
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The SPADE Theory Zl puller a Geld it* in 

Z< The CNLJ theory cantors an the production jjrsinrs model. Although productions 
Are Turing universal, they tend to result in a less structured program organization 
than the linguistic formalisms qT the SPADE theory. In PATH, Band arc transition, 
consisting of a predicate and an action, cm be thought nf as a production, However. 
PATN organizes tltesa productions into local contexts, etch of which consists of the 
arcs exiting from a given node. Not all of the arc productions are present at any 
moment In tlmaj ah arc is present only when the problem solver is at the relevant 
node. In the production system* dlscimed In ffuntait Problem Solving [Ifiwell ft 
Simon ig?£], all of the productions are always present and are tasted in: serial order, 

3. CF!U analyses are based Oil the jtrahlem fteJldiHor ftrcpH. Pursuing an analogy to 
computational linguistics, we define an interpretation of a protocol to be t pent tr#e 
supplemented by semantic and pr agnatic annotation. The parse trea characterizes the 
constituent structure of the protocol. Semantic and pragmatic annotation -- variables 
and assert ions attached to nodes -of tha parse tree -- formal iza the problem description 
and the rationale for particular planning choices. Annotated parse trees closely 
reflect tha local structure of PATH'S linguistic problem solving machinery, leading 
mora directly to inferences regarding individual differences than is evident from 
problem behavior graphs. 

ttuven Brooks [1975] applied the CP1U approach to the programming domain. 
developing a model of coding -- the translation of high level pi ens Into the statements 
of a particular programming language -- end testing the model by analyzing protocols. 
Kls model is a set of production rules whose conditions match the patterns of plan 
elements and whose actions generate code statements. Protocols are analyzed manually, 
with the experimenter attempting to infer the plan which Is then expanded by the 
production system into code paralleling that of the protocol. The processes of 
understanding the problem, generating the plan, and debugging are not formalized* 
SPADE goes beyond this in that it can be used to parse protocols and that the parse 
constitutes t rorraal hypothesis regarding not only the coding knowledge but also the 
planning and debugging strategies employed by the problem solver. 

[Miller a Goldstein 197Gb] provides e detailed example of such analysis being 
performed by hand. The example is a sennent from a protocol several hundred lines long 
in which a high school student uses the Logo turtle tn draw the letters of his name. 



By excraininn the urammar rules present In the derivation, we can readily observe 
various properties of the student's prablan salving such as: reliance on certain 
planning Choices tO the exclusion of others {e.g.. the Student employed iteration „ but 
never recursion); the misuse of certain gptinnai constituents (e.g., a setup was 
Usually included Jin each procedure even when it was unnecessary); and certeln 
situations where his problem solving violates the grwwar and hence is susceptible to 
syntactic errors (e.g., programs were or ten executed before their subprocortures had 
been defined) . 

Just as a context frea grammar is incomplete as a theory Of planning, likewise a 
parse is only a partial analysis of a protocol. The theory of annotation developed in 
the PATN work led us from describing only the syntactic structure to nore complete 
analyses of protocols ■ m interpretation ot a protocol Is the selection) of a 
particular annotated PATN plan derivation. Fig. 9 shows such an analysis of a 
simplified protocol in which i wishingwell program is denned, executed and debugged * 

PAZATN Is a chart-based parser [Kay 13711; Eaplap 1973] being implemented to 
Interpret protocols in terms of PATN ' & annotated plan derivations [Millar ft 
Goldstein 1976d]. It will operate by causing PATN to deviate from its preferred 
approach in response to bottom-lip evidence (Tig. 9). By taking advantage of powerful 
parsing strategies developed in research on speech understanding [Lesser et at, 1*73; 
Paxton ft Robinson 197$], as well as the economical chart representation of ambiguities, 
PAZATH has been successfully hand-simulated on approximately It- Logo protocols. 

PAZATN will operate by matching PATW-genarated plans with protocol data. Two 
charts will be used to represent alternative interpretations. The PLANCHART keeps 
track of the set of plausible SUbgdals which have been proposed by FATtt. Fig. ID shows 
a planchart for a wishingwell In which PATH has proposed two alternative 
decompositions. The structure is a chart because It shares substructures, as 
exemplified by the common solution to the VEIL subgotl pointed to by both wis bin swell 
decompositions. The DATACHAKT records the state of partially completed 
interpretations. Fig. II shows how the datachart links events into the planchart for a 
PAZATK interpretation of the wishlnowell protocol given earlier. 

These charts are grown as follows. First PAZATN requests PATH to generate its 
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nost plausible plan. This plan is inserted into the PLA*lCHAJn\ lhan protocol events 
are analyzed one by one, and matched with subgcials In tho PATN plan. The match is 
recorded in the DATACHART. If no plausible Batches are foimd f PATH 1* esked to 
generate the next nost plausible plan. The Pl.AWCKART is thereby extended. CcmnDn 
sub goals share the seme structure In the chart. 

At first, PAZATN will be inplemented Interactively, with the user -- a 
psychologist analyzing a protocol — directing. PAZATN td Select different PATN plans. 
This follows the : m.: i - v mental Implementation strategy used In two of the CHU protocol 
analyzers [waterman fr Newell 1972 1 Bhaskar & Mwn 1976], PAZATN, even in its early 
interactive stages, should provide strong evidence regarding PATN's adequacy as • 
cognitive theory. 

PAZATN Will also he tested in the SPADE context. Be law Is a hypothetical dialogue 
With SPADE-2, representing the original SPADE-0 augmented by hnth PATH and PAZATN. 

la. Me are now solving Tor a MlSHlKfiWELl. Your pending subgoals are; 
ROOF. PDLX h WELL, and the In tar fates, what now? 

lb, >SGUARI 

2a. OL. I m assuming that WELL ha* been solved hy identifying It 

With tbn previously solved: SQUARE procedure. What HOW7 
Zb. > ... 

PAZATN will increase tha editor's flexibility in handling anblguOUS events, and in 
alleviating what night seem to sou users to be an excess l-ve allocation of tlae and 
effort to tha planning phase- Ultimately we expect STAPES to provide a programing 
environment of value to professional as well as student programmers. 

9, Conclusions 

An outline of tha SPADE theory or design has been presented. One theme H*i been 
the synergism achievable in a multi-faceted research effort. Programs have been 
designed and are being implemented for testing SPADE In educational, psychological and 
AI contexts, The Al dimension involves the development of a spectrum of possible 



theories or cognition. The psychological and educational dimensions supply feedback 
regarding Which part* of a theory ara rejected by competent human problem solvers. 
Furthermore f the psycholofiical dimension would be incomplete ware it riot to address the 
issue of learning h while the educational dimension characterizes the trajectory of th* 
student's cognitive state through tine. 

A Second these has been the exploitation of concepts and algorithms Troo 
computet tan *1 linguistics: grammars, AIN's, derivation treex, search strategies from, 
speech understanding, chart-based parsers. Computational linguistics is also 
responsible Tor suggesting the propitious decomposition of problem solving processes 

into components involving syntactic, semantic atid pragmatic knowledge. 

Ha believe that our unified approach to AI, psychology and education represents a 
new research paradigm ofTering the potential for considerable progress in all throe 
fields. But much remains to be done. Although all of the programs have been designed 
and hand-simulated, only the &PADE-Q editor has bean implemented. Furthermore, while 
SPADE has been applied to Logo graphics, blocks world and elementary calculus problems. 
It has not yet been exercised in enough contexts to prove its generality. If this 
research project succeeds, a new clarity will have been brought to the study of problem 
solving. Even if it rails, the reasons for the failures should provide useful 
insights, In any case, it has already unified the treatment of plans and bugs, a 
s Inn if leant stride Tor a theory of problem solving r 
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